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X.  INTRODUCTION 

The  very  nature  of  pursuit-evasion  problems*  as  continous  dynamics  an 
opposing  objectives#  invites  their  formulation  as  two  person  zero-sum  dif¬ 
ferential  games.  The  study  of  differential  games  emerged  from  the 
pioneering  work  of  Rufus  Isaacs  [1]  in  optimal  pursuit  and  evasion  appliei 
to  tactical  air  combat  problems.  It  was  hoped  that  the  innovative  concep 
of  differential  games  (coined  by  Isaacs  himself)  would  create  an  insight 
into  the  intriguing  problems  of  aerial  "dogfight"  and  eventually  lead  to 
improved  tactics  and  better  design  of  aircraft  and  weapon  systems . 

During  more  than  a  quarter  of  the  century  since  the  first  RAND  repc 
of  Isaacs  on  pursuit  games  in  1951  [2],  numerous  investigations#  dealii 

with  different  aspects  of  this  class  of  problems^  have  appeared  in  the 
literature.  A  considerable  part  of  these  studies,  some  of  them  reviewed 
other  chapters  of  this  Volume  [3*  4)*  was  aimed  at  the  solution  of  air  t 
air  combat  problems.  Unfortunately,  this  extensive  research  effort  has 
only  had  a  limited  impact  on  practical  applications.  This  frustrating  fi 
both  for  scientists  and  military  experts,  can  be  attributed  to  the  follcs 
reasons: 

a.  Xn  the  mathematical  models  used  in  most  analytical  studies,  im 
portant  elements  of  the  "real  world"  situation  have  been  negle 

b.  Not  all  air  combat  problems  satisfy  the  basic  requirements  of 


pursuit-evasion  game  formulation, such  as  fixed  roles  of  the  pi 
and  diagonally  opposed  (zero-sum)  objectives. 
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An  obvious  example  for  (b)  is  the  well  known  "dogfight"  situation  be¬ 
tween  two  fighter  airplanes.  Such  an  engagement,  in  which  both  pilots  wish 
to  assume  an  agressive  role,  calls  for  a  mathematical  formulation  of  a  two- 
target  game  [5].  Moreover,  in  many  air  combat  engagements^  "mutual  kill,” 
which  is  an  equally  disadvantageous  outcome  for  both  participants,  is  a 
possible  outcome .  Consequently,  the  zero-sum  notion  is  not  adequate  for 
such  problems  and  other  concepts  such  as  "preference-ordered  gaming"  [6] 
are  required. 

A  careful  elimination  process  has  lad  to  the  identification  of  some 
types  of  air  combat  problems  well  suited  to  the  pursuit-evasion  game  formu¬ 
lation: 

a.  The  missile-aircraft  engagement. 

b.  Interception  of  an  airplane  not  equipped  with  an  air  to  air 
weapon. 

Both  problems  are  described  by  multidimensional  non-linear  dif¬ 
ferential  equations,  which  leave  no  hope  for  a  closed  form  solution.  The 
numerical  solution  of  the  resulting  non-linear  two  point  boundary  value 
problem  [7,  8]  is  net  only  tedious  and  time  consuming,  but  may  sometimes 
be  even  misleading.  Computation  of  optimal  trajectories,  based  on  local 
necessary  conditions,  cam  be  meaningless  if  unidentified  singular  surfaces 
of  the  game  space  au:e  crossed. 

Important  simplifications  of  the  mathematical  model  can  be  achieved 
by  adopting  some  of  the  following  assumptions :  two-dimensional  motion. 


constant  speeds,  point- mass  approximation,  instantaneous  response,  traject 
linearization.  However,  extreme  care  has  to  be  taken  not  to  use  assumptio 
which  may  eradicate  salient  phenomena  of  the  original  problem. 

This  Chapter  presents  two  techniques  for  solving  pursuit-evasion 
games  of  degree.  Combined  with  a  skillful  mathematical  modeling,  it  has 
the  potential  for  closed  form  solutions  of  near  real  world  air  combat  pro!: 
lems.  In  Section  IX  the  equations  of  motion  of  a  general  air  combat 
oriented  pursuit-evasion  problem  are  given  and  the  corresponding  zero-sum 
perfect  information  differential  }ame  is  formulated.  In  Section  IXX  the 
validity  of  several  simplifying  assumptions  is  discussed  and  guidelines  fc 
appropriate  mathematical  modeling  are  suggested. 

In  the  following  sections,  two  methods  are  proposed  to  solve  proper! 
modeled  pursuit-evasion  games.  Xn  Section  XV  the  formal  solution  of  line* 
differential  games  of  terminal  cost  with  bounded  control  is  presented  and 
applied  to  a  missile-aircraft  end-game.  Xn  Section  V  singular  perturbatii 
technique  is  adapted  for  non-linear  zero-sum  differential  games  and  its 
effectiveness  is  demonstrated  by  the  solution  of  a  simple  aircraft  vs. 
aircraft  problem.  Xn  the  concluding  section,  the  relative  merits  of  the 
proposed  techniques  are  discussed  and  the  possibility  of  their  combinatior 


is  indicated. 
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The  vectors  u,  v  are  the  controls  of  the  pursuer  and  the  evader  respectively 
These  control  variables  (e.g. ,  aerodynamic  surface  deflections,  throttle 
setting,  etc.)  are  bounded,  i.e.,  they  belong  to  the  closed  compact  sets 


u  €  U 
v  €  V 


(6) 


Summarizing  Eqs.(2)-(5),  the  complete  game  dynamics  is  expressed  by  a 
non-linear  autonomous  vector  equation  of  n  components 

z  -  F(z,u,v)  z (tg)  *  zQ  (7) 

where  F  is  continuous  and  differentiable  with  respect  to  its  arguments 
and  z  is  the  state  vector 

*  -  (R.Vp,VE,ap,aE,Cp,CE)  z  €  Rn  (8) 

uniquely  defined  in  a  domain  P  of  Rn.  In  addition  to  the  dynamics,  the 
differential  game  formulation  requires  the  determination  of  (i)  the  infor¬ 
mation  structure;  (ii)  the  criterion  for  game  termination;  (iii)  the  pay¬ 
off  function  (cost) . 


In  the  present  work  it  is  assumed  that  perfect  information  on  all 
components  of  the  state  vector  is  available  to  both  players. 

The  termination  of  the  game  (t^)  is  determined  by  a  closed  subspace 
(the  target  set)  given  by  | 


*tz(tf)]  *  0 


(9) 


If  the  target  set  cannot  be  reached  in  finite  time,  the  game  terminates  when 
the  distance  between  the  players  is  minimal. 

t  =»  arg  min  ^Rlt)l  ^  ^ 

The  coat  function  r  to  be  mjtbnieeC  by  the  evader  and  minimised  by  the 
pursuer,  is  generally  written  as 

j  »  G[z(tJJ  +  /  Uz,u,v,t)dt  (U) 

*0  • 

G  and  L  both  are  continuous  and  differentiable  of  their  arguments. 

The  integral  term  is  of  major  importance  if  control  penalization  is  included. 
The  present  investigation  focuses  on  terminal  pay-off  games  (L  -  0) .  The 
natural  cost  in  pursuit  evasion  games  is  the  time  of  capture  "tf"  defined 
by  Eq. (9)  whenever  the  target  set  can  be  reached.  Alternatively,  the  pay¬ 
off  can  be  the  distance  of  closest  approach  (the  miss  distance)  defined  by 


Eq. (10)  as 


d(tf)  »  min  ^R(t)i 


The  two  game  formulations  can  thus  be  summarized  by 


z  »  F(z,u,v) 


*(V  "  z0 


u  €  U  ,  v  €  v 
t.  -  arg{’|>lz(t)l  *  0} 
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z  =  F(z,u,v)  z(tQ)  =  zo 

u  6  U  ,  v  €  V 

(14 

tf  -  arg  min  *R(t)l 
fc>t0 

J  -  d(tf) 

The  solution  of  a  differential  game  is  a  triplet  consisting  of  an 
optimal  strategy  pair  p*(»)>  e*(>)  and  the  optimal  cost  J*  (the  Value 
of  the  game) .  The  optimal  strategy  pair  has  to  be  selected  from  a  set  of 
admissible  (and  playable)  pairs.  A  strategy  pair  p(»),  e (• )  is  admis- 
sible  if  the  controls 


u(t)  ■  p[z(t),t] 

u  C  U 

v (t)  -  e[z(t),t] 

v  e  v 

(15) 


are  Lebesque  measureable  and  generate  at  least  one  solution  of  the  state 
equation  (7).  Furthermore,  an  admissible  strategy  pair  is  called  playable 
[91  if  it  guarantees  termination  of  the  game. 

The  solution  triplet  has  to  satisfy  the  saddle  point  inequality 


p*,e)  <.  J(z 


(16) 


The  necessary  conditions  to  be  satisfied  by  candidate  solutions  of  an 
autonomous  zero-sum  differential  game  with  terminal  cost,  can  be  stated  as 
follows  [1,  9,  10] s 
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Let  z*  (t)  be  an  optimal  trajectory  and  assume  that  J*(z,t),  the 
Value  of  the  game,  is  smooth  along  it;  then  there  exists  a  continuous  vector 
function  A (t)  and  a  Hamiltonian,  defined  by 

H(z,A,u,v)  =  XT  F(z,u,v)  (17) 

satisfying  the  adjoint  equation 

*<*>  •  -(H) .... 

and  the  transversality  conditions  which,  subject  to  the  definition  in  Eq.(9), 

i' 

takes  the  form 

A  (t^.)  *v  grad  tji  [z(tf)  ]  :  V  >  0  (19) 

For  cases  where  game  termination  is  determined  by  reaching  the  distance 
of  closest  approach,  which  is  also  the  pay  off,  the  transversality  condition 

i 

is  expressed  by 

A  (tf )  -  grad  d(tf)  (20) 

Moreover,  the  Hamiltonian  also  satisfies 

Min  H( z*,A,u,v*}  *  Max  H(z*,A,p*,v)  *  0  (21) 

u  €  O  v  €  V 

The  optimality  of  the  candidate  strategy  pair,  obtained  from  (21),has 
to  be  established  by  sufficiency  conditions,  presented  in  Refs.  [9,10]. 


The  necessary  conditions  indicate  that  in  order  to  attain  a  candidate 
solution  of  a  game,  a  non-linear  two-point  boundary  value  problem  of  the  order 
2n  has  to  be  solved. 

In  the  original  pursuit-evasion  game  described  in  this  section  the  number 
of  state  variables  (Eqs.  (2)- (5))  is  very  large  since  R,Vp,Vg,ap,aE  are 

all  three-dimensional  vectors  and  Cp,CE  ®ey  even  have  more  components . 

The  modelling  effort,  discussed  in  the  next  section,  is  aimed  at  reducing 
the  number  of  state  variables  while  retaining  a  truthful  representation  of  the 
prominent  features  of  the  original  problem. 


IZZ.  MODELING  CONSIDERATIONS 

The  key  to  a  useful  solution  of  any  complex  problem  is  the  skillful 
choice  of  the  simplest  possible  model  which  preserves  the  salient  system  prop¬ 
erties  under  investigation.  Guidelines  to  this  effect  are  suggested  by  the 
following  critical  discussion  of  some,  frequently  used,  assumptions. 

A.  Simplifying  Assumptions  _ 

1 .  Two-dimensional_motion 

Restricting  the  motion  of  the  players  within  a  plane, results  in  a 
reduction  in  the  number  of  the  state  variables  (at  least  by  5) .  In  many  cases 
the  2-D  analysis  provides  an  initial  insight  into  the  problem.  However,  some 
inherent  characteristics  of  the  original  3-D  problem  may  be  absent  in  a  2-D 
model.  Therefore,  results  of  2-D  solutions  must  be  carefully  examined  if  a 
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"real  world"  3-D  Interpretation  is  required.  This  point  can  be  well  illustrated 
by  the  example  of  optimal  missile  avoidance.  The  first  used  2-D  model  [11] 
led  to  discover  the  "bang-bang”  nature  of  the  optimal  maneuver,  but  only  a 
later  3-D  analysis  [12]  could  define  its  optimal  direction,  which  is  perpen¬ 
dicular  to  the  plane  of  collision. 

2 .  Point-mass  aggroximation 


With  this  approximation  vehicle  dimensions  and  rotational  degrees  of 
freedom  are  disregarded.  The  vehicle  is  represented  by  its  center  of  gravity. 
The  complex  non-linear  dynamics  of  the  vectors  Cp, 

by  linear  differential  equations  and  often  represented  only  by  first  order 
time  constants  as: 


C£  in  (5)  can  be  replaced 


(22) 


The  disregard  of  vehicle  dimensions  has,  however,  a  serious  limitation.  If 
the  distance  of  closest  approach  is  of  the  same  order  of  magnitude  as  the  size 
of  the  airplane  involved,  the  very  concept  of  closest  approach  defined  by  (12) 
may  loose  its  meaning.  However,  for  a  case  resulting  in  very  small  or  very  large 
miss  distances,  the  point  mass  approximation  is  a  useful  and  justifiable  as¬ 


sumption. 


11 


3.  Instantaneous  control  response 

This  frequently  used  assumption  disregards  the  time  lag  in  the  control 
inputs  by  setting  Tp  =  =  0  in  Eq.  (22) .  This  assumption  leads  to  the 

conclusion  that,  for  pursuer's  speed  and  maneuver  advantages  :  Vp  >  VE  »  ap  i  aE  /> 
a  "point  capture"  would  be  possible  [13,  14] .  This  conclusion  is  disapproved 
by  the  move  complete  model.  Thus,  in  problems  where  miss  distance  calcula¬ 
tion  is  important,  the  assumption  of  "instantaneous  response"  is  inadequate. 

4.  Constant  speed 

This  assumption  rarely  represents  physical  reality.  It  can,  however, 
be  justified  in  problems  of  short  duration,  where  the  effect  of  velocity 
change  is  negligible  and  for  vehicles  in  which  the  longitudinal  component  of 
the  acceleration  is  much  smaller  than  the  lateral  ones.  In  such  cases  constant 
speed  models  give  a  fairly  good  description  of  the  main  phenomena. 

5.  linearization 

In  some  pursuit-evasion  problems  there  exists  a  reference  trajectory 
allowing  linearization  of  the  originally  non-linear  kinematics.  Collision 
course  (see  Fig.  2)  is  an  example  for  such  situation.  Trajectory  lineariza¬ 
tion  is  justified  only  if  the  total  direction  change  during  the  engagement 
is  not  too  important. 

A  valid  trajectory  linearization  combined  with  the  constant  speed  as¬ 
sumption  allows  to  describe  the  relative  motion  by  a  set  of  linear  differen¬ 
tial  equations.  Since  in  this  case  'he  velocity  components  along  the  reference 
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trajectory  are  almost  constant,  changes  in  this  direction  can  be  expressed  as 
a  function  of  the  time,  resulting  in  a  further  reduction  in  dimensionality. 
Moreover,  in  such  problems  the  capture  time  "t^"  can  be  determined. 

B.  Model  Formulation 

Any  deterministic  pursuit-evasion  process  can  be  divided  into  three 
phases:  the  initial  "acquisition"  phase,  the  main  "pursuit"  phase  and  the 
"end-game."  In  the  main  pursuit  phase  the  distance  of  separation  between  the 
players  is  reduced  and  the  state  of  the  game  approaches  the  target  set.  If 
the  initial  conditions  of  the  engagement  are  unfavourable  to  such  "pure  pursuit"^ 
the  "acquisition"  phase  becomes  important.  This  phase  is  characterised  by 
significant  directional  changes  of  the  trajectories.  As  the  game  nears  its 
termination,  the  attention  of  the  "players"  is  focused  cn  the  conditions  im¬ 
posed  by  the  terminal  constraints.  Consequently,  the  optimal  strategies  of  the 
"end-game"  cam  be  very  different  from  the  ones  used  in  other  phases  of  the  game. 

These  observations  indicate  that  the  best  mathematical  model  is  not  neces¬ 
sarily  the  same  for  all  phases  of  a  pursuit-evasion  game.  As  examples,  let 
us  examine  the  two  air  combat  problems,  well  suited  for  zero-sum  differential 
game  formulation,  indicated  in  the  Introduction.  1 

1.  Missile_vs«i_Aircraft_Game  \ 

The  majority  of  such  engagements  (excluding  the  type  of  "dogfight  mis¬ 
siles,"  to  be  mentioned  later)  can  be  characterized  by: 


/' 


/ 
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a.  The  pursuer  has  a  definite  advantage  both  In  speed  ;  vp  >  VB 
and  maneuverability^  a^  >  a£  . 

b.  The  launching  platform  (either  airborne  or  ground  based) 
provides  generally  favourable  initial  conditions  for  the  pursuit. 

c.  The  outcome  of  the  engagement  can  be  measured  by  the  "miss 
distance"  (distance  of  closest  approach) ,  thus  termination  of  the  game  is 
guaranteed. 

In  such  engagements,  the  emphasis  is  obviously  on  the  "end- game"  and  as 
a  consequence  of  (b)  the  acquisition  phase  can  be  neglected.  The  mathematics 
model  of  this  problem  can  be  based  on  trajectory  linearization  as  well  as  on 
constant  speed,  point  mass  approximations,  but  cannot  assuae  instantaneous 
control  response  of  the  pursuer.  This  mathematical  model,  both  in  a  2-D  or 
a  3-0  version,  yields  linear  time  dependent  differential  equations  of  motion. 
Let  us  remark,  however,  that  the  validity  of  trajectory  linearization  and  the 
assunption  neglecting  vehicle  size  has  to  be  verified  "a  posteriori." 

For  future  dogfight  missiles  of  the  ASRAAM  type,  the  acquisition  phase 
is  of  major  importance.  To  describe  this  phase , a  non-linear  model  must  be 
used. 

2.  Aircr^f c_vs. _Air craft  Interception  Game 

In  this  engagement  the  lack  of  air  to  air  weapon  forces  one  of  the  planet 
to  assume  the  role  of  the  evader.  The  pursuing  fighter  may  or  may  not  have 
speed  or  maneuverability  advantage  relative  to  his  opponent.  Initial  conditio 
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nay  or  may  not  be  favorable  for  interception.  As  a  consequence,  in  this 
game  the  acquisition  phase  is  of  major  importance.  The  interception  will 
be  successful  if  the  pursuer  can  reach  the  evader  at  a  distance 
determined  by  the  "firing  envelope”  of  its  weapon  within  a  finite  time  inter¬ 
val.  If  terminationi  as  defined  above!  is  possible,  the  natural  cost 
function  of  this  game  is  the  time  of  capture. 

Since  the  maximum  firing  range  of  modem  air  to  air  missiles  largely 
exceeds  the  radivs  of  turn  of  combat  airplanes,  the  "end-game"  phase  in 
such  engagements  is  hardly  noticed. 

In  the  aircraft  interception  game  there  is  no  requirement  for  accurate 
miss  distance  calculation.  Consequently,  the  assumptions  of  instantaneous 
control  response  and  point  mass  approximation  can  be  adopted.  Since  the 
very  nature  of  the  acquisition  phase  does  not  allow  trajectory  lineariza¬ 
tion,  game  dynamics  remains  non-linear.  The  validity  of  constant  speed  and 
2-D  models  strongly  depends  on  problem  parameters  and  has  to  be  examined 
separately.  These  two  assumptions  seem  to  be  tied  together.  In  previous 
works  [15,16],  it  has  been  shown  that  the  optimal  pursuit  evasion  game  of 
constant  speed  vehicles  is  confined  to  a  plane.  This  result  may  not  be 
true  for  variable  speed  airplanes  even  if  the  initial  conditions  are  two- 
dimensional. 
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C.  Selection  of  Solution  Techniques 

Pursuit-evasion  games  of  valid  linear  mathematical  models  can  be  analyse< 
by  the  powerful  methods  of  linear  differential  game  theory.  Though  attention 
in  the  past  has  been  focused  on  linear  games  with  quadratic  pay-off  functions 
and  unbounded  controls  (17,  18,  19,  20],  examples  of  terminal  cost  linear 
games  **■*«•,»  hard  bounded  control  were  also  solved  [21,  22]. 

In  Section  IV  such  latter  version  is  applied  to  solve  the  missile  vs. 
aircraft  end-game  with  realistic  dynamics.  In  the  sequel,  implementation 
of  the  results  for  missile  guidance  as  well  as  for  missile  avoidance  are 
discussed  and  the  validity  of  the  linear  model  is  examined.^ 

I 

For  problems  where  trajectory  linearization  cannot  be  justified  a  non¬ 
linear  two-point  boundary  value  problem  remains  to  be  solved.  Exact  solu¬ 
tions  in  closed  form  exist  only  for  problems  with  very  low  dimension  [1, 

23,  24]  using  oversimplified  mathematical  models.  For  a  practical  applica- 

i 

tion,  however,  an  approximate  solution  of  a  near  real  world  model  seems  much 
more  attractive.  In  recent  years  several  non-linear  two  point  boundary 
value  problems  originating  in  optimal  control,  including  problems  of  air¬ 
craft  performance  optimization,  have  been  solved  using  the  approximation 
technique  of  singular  perturbations  (SPT)  [25,  26,  27,  28].  Linear  dif¬ 
ferential  games  of  high  dimensions  were  also  treated  by  the  same  method  [29, 
30,  31].  In  a  recent  study  [32],  it  was  proposed  to  apply  the  method 
of  singular  perturbations  to  non-linear  differential  games. 

In  Section  V  the  basic  notions  and  principles  of  SPT  are  outlined  and 
the  application  for  non-linear  pursuit-evasion  games  is  discussed.  The 
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merits  of  the  proposed  approximation  technique  is  demonstrated  by  a  simple 
(2-0,  constant  speed)  example .  The  method  of  SPT  has,  however,  straight¬ 
forward  extension  for  more  realistic  (variable  speed,  3-0)  models. 

IV.  LINEAR  DIFFERENTIAL  GAMES  WITH  BOUNDED  CONTROLS 

Linear  differential  games  (LOG)  have  been  extensively  investigated  in 
the  Inst  15  years  on  both  sides  of  the  iron  curtain  (17-20,  34-38] .  However, 
the  potential  Gf  LDG  technique  to  solve  realistic  pursuit  problems  (i.e., 
games  of  terminal  cost  with  bounded  controls)  was  only  recently  realized 
[21,  22,  39-41].  In  order  to  demon.. '■rate  its  effectiveness  as  an  analyti¬ 
cal  tool,  the  LDG  technique  will  be  applied  in  this  section  to  solve  the 
missile  vs.  aircraft  engagement  described  in  III.B.l.  The  solution  has  a 
clear  geometric  interpretation  which  enables  to  discuss  the  implementation 
of  the  optimal  strategies  as  well  as  the  validity  of  the  linear  model. 

A.  Formulation  of  the  Missile  vs .  Aircraft  Game 

Based  on  the  description  of  such  an  engagement  given  in  the  previous 
section  (III.B.l)  the  following  set  of  assumptions  is  adopted: 

1)  Both  pursuer  and  evader  are  considered  as  point-mass  vehicles. 

2)  The  speed  of  each  vehicle  is  constant,  the  pursuer  being  the  faster 
<VVE  >  1). 

3)  The  relative  motion  is  three-dimensional  (See  Fig.  3). 
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4)  Gravity,  having  no  effect  on  the  relative  trajectory,  is  neglected. 

5)  The  initial  conditions  of  the  pursuit  are  near  to  a  collosion  course 
(See  Fiv-2). 

6)  The  relative  trajectory  can  be  linearized  around  the  initial 
line  of  sight  vector. 

7)  The  performance  index  of  the  problem  is  the  miss  distance  (distance 
of  closest  approach) . 

8)  There  exists  per7  >ct  (complete  and  instantaneous)  information  on  the 
state  variables  \nd  the  parameters  of  the  problem. 


9)  The  lateral  acceleration  commands  of  both  vehicles  are  bounded  by 
circular  vectograms  perpendicular  to  the  respective  velocity  vectors 
(a^/a^  >  1) .  This  assumption  will  be  slightly  modified  in  the  course 
of  the  solution. 


10)  The  pursuer's  response  to  its  acceleration  command  is  approximated 
by  single  tin.  consist  Tp  . 

11)  Evader  dynamics  can  be  approximated  by  a  first  order  time  constant 


Assumptions  2,  5,  and  6  lead  tc  a  set  of  linear  differential  equations. 
Moreover,  as  a  consequence  of  the  linearization,  the  relative  motion  in  the 
line  of  sight  direction  (the  X  axis)  is  of  constant  speed  and  the  duration 


of  the  game  tf  is  determined 


The  state  vector  of  this  problem  has  eight  components 


z  =  col  {y, 


z,z,zp,zE} 


(23) 


where 


Y  =  YP 

z  -  Zp 


(24) 


The  dynamics  to  be  considered  is  perpendicular  to  the  line  of  sight.  In 
this  coordinate  system  the  circular  vectorgram,  perpendicular  to  the  respective 
velocity  vectors  (see  Ass.  9),  becomes  eliptic  as  depicted  in  Fig.  4.  Ac¬ 
cordingly,  the  admissible  control  sets  u  and  v  have  the  form 


with 


u  =  {u  ;  uTRu  <.  ap} 

v  =•  {v  s  vTSv  <  a^l 
£» 


and 


1/ cos  Xp(°)  0 


1/ cos  XE<0)  0 


(25) 


(26) 


The  game  dynamics  can  be  now  describe*,  by 
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z  *  Az  +  Bu  +  cv 
u  €  U  ,  v  €  V 

with 


(27) 


The  pay-off  of  the  game  is  defined  by  (see  Ass.  7) 
J  -  *D  z(t£)* 

with 


(78) 


It  is  required  to  find  among  all  admissible  strategies  {p(*),e(*)}  / 
such  that  u(t)  *  p[z(t),t],  and  v(t)  -  eCzttJtl^—  an  optimal  pair 
{p*  (• ) *e*  (•) }  satisfying  the  saddle  point  inequality 

A 

J(z,t,p*,e)  <.  J(z,t,p*,e*)  *  J*(z,t)  <.  J(z,t,p,e*) 


(29) 
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Equations  (27)  and  (28)  can  be  simplified  by  using  the  following  transforma¬ 
tion: 


y (t)  *  D  $(tf,t)z 


8(t)  -  D  *(tf,t)B 


(30) 


C(t)  *  D  4>(tf,t)C 


where  #(tf,t)  is  the  transition  matrix  of  the  linear  system  z  ■  Az  sat¬ 


isfying 


—  $(tf,t)  »  -$(tf,t)A 


(31) 


By  Eq.  (30)  and  (31)  the  original  formulation  of  Eq.  (27)  and  (28)  becomes 


y  »  8(t)u  +  C(t)v 
u  €  U  v  6  V 


y(0)  -  y„ 


(32) 


and 


J  “  ly  (tf)  I 


(33) 


The  new  2-D  variable  y  earn  be  interpreted  as  the  vector  of  the  "pre¬ 
dicted  miss  distance."  In  this  particular  problem 


$(tf,t)=$(tf-t,o)=4>(6)  = 


(0) 

0 

0 

*1 (0) _ 

(34) 


”8"  being  the  normalized  "time  to  go?  defined  as 
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B.  Solution  of  the  Game 

Tte  Hamiltonian  of  the  transformed  game  is 
H(y,X,t)  *  XTv  -  XT[B(t)u  +  C(t)V] 
where  X(t)  has  to  satisfy 
dA  3H 

- >  as  —  *  U 

dt  3y 

X(tf)  -  grad  J  -  iy(yr 


(40) 


(41) 


(42) 


Assming  that  X(t)  is  continuous  (this  hypothesis  must  be  verified) 


Eqs.  (41)  and  (42)  yield 


X(t) 


y(V  A  * 

17<VT  5 


(43) 


|  being  a  constant  unit  vector  along  each  optimal  trajectory. 

The  candidate  optimal  strategy  pair  can  be  new  determined  by 
Min  |T8(t)u  -  !TS(t)p* 

uGu  ««> 

Max  |TC(t)v  -  |TC(t)e* 
v  €  V 


yielding 


L. 
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indeed  optimal,  sufficiency  conditions  have  also  to  be  satisfied.  In  previous 
studies  [9,  10]  it  was  proven  that  the  existence  of  smooth  isocost  surface.*! 
("tubes")  generated  by  the  candidate  solution,  guarantees  saddle  point 
optimality. 


Analysis  of  the  present  solution  has  shown  [39],  that  in  soeie  subregions 
of  the  game  space  the  isocost  surfaces  are  only  piece-wise  smooth.  A 
detailed  analysis  of  such  singular  phenomena,  induced  by  the  eliptic  vecto- 
grams  [40],  is  out  of  the  scope  of  the  present  discussion.  Moreover,  the 
singularity  can  be  avoided  by  a  slight  modification  of  the  admissible  control 
set.  Let  us  replace  Egs.  (25)  and  (26)  by 


a  /  T  .  .2 i 

u  *  iu  :  u  u  <  ap/ 

A  /  T  .  a2i 

V  -  iu  s  v  V  <  a  j 
E 

with 


(50) 


S  -  a  cos  v  (0)  >  a  -  ap  (51) 

P  P  *P  “*  E  E 

resulting  in  circular  vectograms  for  both  players.  This  formulation  is  an 
exact  one  for  "head  on"  or  "tail  chase"  engagements,  where  cos  Xp(°)  * 
and  represents  a  slightly  pessimistic  assumption  for  the  pursuer  for  other 
initial  geometries.  Adopting  this  modification,  the  solution  of  the  game 
becomes 


/ 
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J*(y,9) 

m 


c  with 
m 


c 

m 


Vp 


(59) 


A  cross  section  of  such  "minimal  tube"  is  shown  in  Fig.  5.  The  values 
~  A  2 

of  0  and  c  ■  c  /a_T_  are  depicted  in  Figs.  6  and  7  as  the  function 
s  m  m  E  P 

of  the  maneuver  ratio  aE/ap  and  the  time  constant  ratio  Tg/Tp  • 

If  inequality  (58)  is  satisfied,  the  (y,0)  game  space  can  be  de¬ 
composed  to  iP  and  P^  by  defining: 

P°  =  the  interior  of  the  "minimal  tube"  for  8  >  0  and 

8 

P^  as  its  complement. 

The  solution  of  the  game  (the  optimal  strategy  pair  and  the  Value) 
are  given  in  P*  by  Eqs.  (52) -(54).  In  tP  any  arbitrary  admissible 
strategy  pair  is  optimal  end  the  Value  is  constant,  given  by  (59) . 

If  inequality  (58)  is  not  satisfied,  point  capture  can  be  guaranteed 
by  the  pursuer  in  a  part  of  the  state  space.  For  such  case  the  decomposi¬ 
tion  of  the  state  space  is  slightly  modified  by  defining  P°  as 

P°’  -  { (y, 0)  :  ly (6) I  <  ly  (0)*  -  0(0)  -  8(0)}  (60) 

s 

The  optimal  strategies  in  this  domain  are  arbitrary  and  the  Value  is  zero. 

Outside  this  region  (ly(0)l  >  ly  (9)1)  the  optimal  solution,  expressed 

—  s 

by  Eqs.  (52) - (54) ,  remains  valid. 
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C.  Implementation  and  Validity 

The  closed  form  solution  of  the  missile  vs.  aircraft  game  obtained 
in  the  previous  subsection  is  based  on  perfect  information  (Ass.  Mo.  8) 
inlying  Jiat  all  state  variables  can  be  instantaneously  and  accurately 
measured  by  both  players.  Validity  of  this  assumption  has  a  major  impact 
on  the  implementation  of  the  optimal  strategies. 


The  components  of  the  "predi  ted  miss  distance"  vector  y  in  Eq.  (38) 
are  composed  of  3  parts  of  different  origin.  The  first  two  terms  form  the 
respective  components  of  the  "zero  effort  miss,”  proportional  to 
the  turning  rate  of  the  line  of  sight,  a  . 


Y  +  T  0Y  -  v  [t  8)  2  a 
p  c  P  y 

z  +  T  0Z  -  V  (T„0)2  a 
P  c  P  z 


(61) 


Vc  being  the  closing  speed  of  the  interception  defined  by  (see  Pig. 2), 


Vc  *  Vp  cos  Xp<0>  -  VB  cos  XE(0)  -  const.  (62) 

The  other  two  terms  of  Eq.  (38)  are  the  properly  weighted  lateral 
accelerations  of  the  players.  Acceleration  can  be  easily  measured  onboard 
a  flying  vehicle  but  its  estimation  from  another  moving  platform  is  a  very 
complicated  and  tedious  task.  Based  on  this  preliminary  observation,  the 
implementation  of  the  analytics'',  results  of  the  previous  subsection  will 
be  separately  discussed  for  applications  in  optimal  missile  guidance  and 


missile  avoidance. 
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1.  Optimal  Missile  Guidance 

Measurements  of  the  line  of  sight  rate  a  ,  as  well  as  the 
missile's  own  lateral  accelerations  has  been  of  common  practice  in  almost 
every  guidance  system.  Estimation  of  target  acceleration  by  Kalman  filters 
has  been  also  contemplated  for  some  modern  weapon  systems  in  order  to  im¬ 
prove  their  performance.  A  recent  study  [43],  however,  indicated  that  such 
"optimal"  guidance  schemes  are  very  sensitive  to  parameter  variations  and 
may  not  be  cost  effective. 

Renouncing  measurement  (or  estimation)  of  target  acceleration  by  the 
pursuer  is  equivalent  to  assume  instantaneous  evader  dynamics.  Substituting 
Te  *  0  into  Eqs.  (57)  and  (58)  yields  (41] 


r  > 

E  0 


(63) 


(64) 


These  results  represent  the  "worst  case"  analysis  for  the  pursuer. 

The  proposed  strategies  can  serve,  however,  as  directive  for  the  optimal 

guidance  law  synthesis.  The  existence  of  a  "minimal  tube"  and  consequent 

decomposition  of  the  (y,6)  state  space  are  of  major  significance.  The 
_0 

region  V  ,  which  is  dominated  by  the  pursuer,  is  characterized  by  small 

deviations  from  collision  geometry  and  not  too  short  pursuit  times  (8  >  6  ) . 

80 

Most  trajectories  start  in  this  region  (see  Ass.  No.  5)  and  for  all  of  them 
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the  predicted  miss  distance  can  be  reduced  to  zero  at  6-9  ,  against 

■o 

any  admissible  evasive  maneuver,  using  an  arbitrary  admissible  pursuer 
strategy.  Thus,  the  optimal  guidance  law  can  be  selected  according  to 
sane  other  practical  design  considerations  (as  minimal  control  effort 
for  example)  and  not  by  miss  distance  minimization.  A  time-varying  linear 
feedback  control  law  proposed  in  a  previous  2-D  study  [22]  can  be  an  at- 
tractive  choice.  (Note  that  the  computation  of  the  "time  to  go,"  which 
requires  measurements  or  estimation  of  range  and  range  rate,  is  a  rela¬ 
tively  simple  task  for  many  guided  missile  systems.)  The  guaranteed  miss 

distance  c  in  Eq.  (64)  can  be  reduced  by  increasing  missile  maneuver- 

m. 

ability  and  decreasing  its  time  of  response.  The  final  value  of  c  shoul< 

XU  r 

serve  as  a  guideline  for  warhead  design. 

The  conclusions  of  the  analysis  for  missile  avoidance  are  not 
encouraging.  In  cases  which  involve  well  designed  missiles, inequality  (58) 
is  generally  not  satisfied.  If  the  guidance  system  is  capable  of  measuring 
or  estimating  the  accelerations  of  the  evader  with  adequate  precision,  zero 
miss  distances  are  predicted  for  most  initial  conditions  against  all  evasivt 
maneuvers,  (in  such  cases  aircraft  survivability  can  be  enhanced  only  by 
denying  information  from  the  missile) . 

Nevertheless,  if  it  is  known  that  the  missile  guidance  law  does  not 
rely  on  estimation  of  evader  acceleration,  the  analytical  solution  of  IV  B 
provides  some  clue  for  a  practical  evasion  strategy.  The  proposed  strategy 


30 


consists  of  a  well  timed  "hard"  maneuver  in  an  optimal  direction  and  does 

not  require  any  elaborate  measurement.  According  to  the  concept  of  the 

"minimal  tube,"  maneuvering  at  9  >  9  in  P°  does  not  contribute  to  the 

final  outcome,  because  the  pursuer  can  guarantee  zero  predicted  miss 

distance  at  9  =  9  .  At  that  point  (y  *  0,  0  «■  9  )  ,  however,  evader 

s0  S0 

maneuvers  become  effective.  The  optimal  direction  of  the  maneuver  can 
be  determined  by  inspection  of  Eq.  (47)  and  the  original  elliptical  vecto- 
grams  in  Fig.  4.  It  can  be  directly  concluded  that  the  miss  distance  is 
maximized  by 

S*T  =  (0,  ±1)  (65) 

indicating  that  the  optimal  maneuver  direction  is  perpendicular  to  the 
plane  of  collision  (see  Fig.  2)  which  is  determined  by  the  initial  condi¬ 
tions. 

The  timing  of  this  terminal  maneuver  need  not  be  very  accurate.  It 
is  sufficient  to  start  the  "hard"  turn  at  some  0  >  9  in  order  to 

so 

guarantee  a  miss  distance  of  c  given  by  Eq.  (64) .  In  this  sense, 

mo 

the  proposed  very  simple  avoidance  strategy  is  optimal. 


3.  Validity  of  the  Linearization 

The  main  limitations  of  the  analytical  solution  presented  in 
IV  B  lie  in  the  assumptions  of  perfect  information  and  trajectory  lineariza¬ 
tion.  The  implications  of  partial  information  were  considered  in  the 
previous  subsections  dealing  with  implementation  of  the  optimal  strategies. 
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Other  aspects  of  information  imperfections  are  discussed  in  some  detail 
in  a  recent  paper  [39].  In  the  following,  the  validity  of  trajectory 
linearization,  which  is  essential  to  apply  LOG  techniques,  is  examined. 

Linearization  is  based  on  the  hypothesis  (see  II.A.5)  that  the 
directional  changes  during  the  engagement  are  not  important.  This  assump¬ 
tion  is  valid  [41]  if  two  conditions  are  satisfied: 

a.  The  direction  change  of  the  evader  during  the  period  of  one  rimm 
constant  of  the  pursuer,  defined  also  as  the  "dynamic  similarity  parameter" 
[44]  of  the  pursuit -evasion  problem 


is  small. 


b.  The  optimal  solution  does  not  predict  excessively  long  maneuvers  i 
any  direction. 


The  first  condition  can  be  observed  before  the  linearization  is  adopts 
The  second  one,  however,  requires  an  "a  posteriori"  verification. 

For  trajectories  starting  outside  the  "minimal  tube"  the  second  condi¬ 
tion  is  generally  not  fulfilled.  In  this  region  (t^)  ,  constant  direc 
tion  maneuvers  are  optimal  (see  Eqs.  (45)  or  (52)),  leading  to  significant 
changes  in  interception  geometry,  unless  the  duration  of  the  engagement  is 
very  short.  For  initial  conditions  in  V°  the  validity  of  the  lineariza¬ 
tion  depends  on i the  actual  strategy  selected  by  the  evader.  Either  a  pass! 
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evader  behavior,  based  on  minimum  control  effort  consideration,  or  periodical 
evasive  maneuvers  [451  will  maintain  the  initial  geometry  and  consequently 
justify  linearization.  The  short  terminal  maneuver  initiated  at  0  * 

(see  IV. C. 2)  induces  only  minor  direction  changes. 

It  can  be  thus  summarized  that  the  linearized  kinematic  model  provides 
a  valid  description  of  the  missile  vs.  aircraft  "end-  game"  and  engagements 
starting  (and  remaining)  near  to  the  initial  collision  course.  For  other 
initial  conditions  the  original  non-linear  trajectory  equations  have  to  be 
solved. 


V.  SINGULAR  PERTURBATION  TECHNIQUE  FOR  NON-LINEAR  PURSUIT-EVASION 
GAMES. 


A.  Preliminaries 

The  technique  of  singular  perturbations  (SPT)  has  been  successfully 
used  in  approximate  solutions  of  non-linear  optimal  control  problems  [25-27]. 
Hie  first  attempt  to  apply  the  same  technique  in  non-linear  zero-sum  dif¬ 
ferential  games  is  a  very  recent  one  [32] .  In  this  section  the  basic 
principles  of  the  method  and  its  application  for  non-linear  pursuit-evasion 
games  are  briefly  summarized  and  illustrated  by  an  example. 

A  dynamic  system  has  a  singularly  perturbed  structure  if  it  involves 
a  small  parameter  e  in  such  a  way  that  the  state  vector  z  €  Rn  can  be 
decomposed  to  subvectors  x  €  Rn  and  y  €  Rn-Ia 


and  state  equation 
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z  »  F(z,u,v,E)  ;  z(tQ)  *  zQ 


can  be  written  as 


x  *  f(x,y,u,v,£)  ;  x  (tQ)  -  xQ 


£y  «  g(x,y,u,v,E)  j  y(tQ)  *  yQ 


The  existence  of  the  singular  perturbation  parameter  E  is  always 
linked  to  the  time  scale  separation  of  the  state  variables.  If  the 
functions  f{*)  and  g{*>  have  the  same  order  of  magnitud*  it  is  clear 
that  the  rate  of  change  of  y  is  much  faster,  than  the  variations  of  x. 
There  are  however  many  non-linear  dynamic  systems  of  well  known  time 
scale  separation  between  "fast"  and  "slow”  variables,  for  which  the 
direct  identification  of  the  small  parameter  is  a  complex  task.  In 
such  problems  (e.g.:  aircraft  performance  optimization  [25-26])  the 
singular  perturbation  parameter  can  be  introduced  artificially.  The 
technique  used  in  such  "forced"  singular  perturbation  (FSPT)  problems 
is  similar  to  the  genuine  SPT. 


If  the  singular  perturbation  problem  is  "well  posed",  the  solution 
of  the  reduced  order  system  (where  e  -  0)  is  a  good  approximation  of 
the  exact  solution  of  the  original  problem ,  however ,  it  generally 
cannot  satisfy  the  end  conditions  of  the  fast  variable.  This  discrep¬ 
ancy  can  be  bridged  by  "boundary  layer"  solutions  obtained  using  a 
stretched  time  scale 


\.  '■ , 

\N  : 


y 


T  * 


(70) 


e 


For  a  uniformly  valid  approximation,  the  boundary  layer  solution 
must  be  asymptotically  stable  and  should  match  the  reduced  order  solu¬ 
tion.  If  this  condition  is  satisfied,  a  uniformly  valid  additive  com¬ 
posite  solution  can  be  synthetized  and  serve  as  a  zero-order  approxima¬ 
tion.  If  more  accuracy  is  required,  the  variables  can  be  expanded  into 
asymptotic  power  series  and  higher  order  terms  can  also  be  taken  to 
account. 

The  most  attractive  feature  of  SPT  is  that, if  the  original  problem 
can  be  decomposed  (assuming  complete  time  scale  separation)  to  succes¬ 
sive  boundary  layers  with  a  single  active  state  variable  in  each,  the 
solution  is  obtained  in  a  feedback  form.  Moreover,  in  such  a  scheme 
genuine  SPT  and  FSPT  yield  identical  zero -order  results. 

I 

I 

Such  feedback  solutions  have  a  great  potential  for  real  time  air¬ 
borne  applications.  The  method  of  complete  time  separation  is  very 

i 

effective  for  problems  of  initial  boundary  layers.  Recent  studies 
[46,  47]  identified  difficulties  in  obtaining  feedback  solutions  for  sin¬ 
gular  perturbation  problems  of  terminal  boundary  layer.  Structures  of 
mathematical  models  for  which  SPT  fails  were  also  observed  [48] . 

The  application  of  SPT  to  differential  games  is  more  than  a  mere 
extension.  Singularly  perturbed  linear  differential  games  were 
investigated  in  the  past  [29  -31],  revealing  the  problem  of  "ill  posedness" 
Though  it  has  been  shown  [29],  that  singularly  perturbed  zero-sum  linear 


/ 


/ 


35  - 


differential  games  are  well  posed,  no  such  proof  has  been  given  for  non¬ 
linear  games.  Application  of  SPT  to  non-linear  zero-svm  differential 
games  is  discussed  in  detail  in  a  recent  study  [32.1 .  In  the  following 
subsections  the  main  results  of  the  investigation  are  outlined  and 
implemented  for  a  non-linear  pursui t-evasion  game. 

B.  Singularly  Perturbed  Non-Linear  Differential  Game 
1.  Original  Game 

Consider  a  singularly  perturbed  autonomous  non-linear  dynamic 
system,  controlled  by  two  competing  players  (P,  E) ,  described  by 

x  «  f (x,y,u,v,e)  ;  x(tQ)  -  xQ  (71) 

ey  «  g(x,y,u,v,e)  t  y "  yo  l72) 

e  being  a  small  parameter,  x  €  Ra  ,  y  6  Rn-in  ,  x+y  €  Rn  . 

It  is  required  to  find  -in  optimal  strategy  pair  p*  (»,E)^e*  (•,£) , 
selected  from  the  set  of  admissible  and  playable  pairs  p(»,e) ,e(*,e)/ 
such  that 


u(t,e)  -  p{x(t) ,y (t) ,e}  u  €  U 

v(t,e)  ■  e{x(t) ,y(t) ,e}  v  6  V 


(73) 


trams fers  the  system  from  the  given  initial  conditions  to  a  terminal 
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manifold 

f[*(t,).e]  =  0  (74) 

optimizing  [P  is  the  minimi zer  and  E  is  the  maximizer)  the  terminal 
cost  function 

J=G[x(tf),e)  (75) 

Note  that  in  this  game  both  the  terminal  manifold  and  the  cent 
depend  only  on  the  "slow"  components  of  the  state  variable. 

Assumption  1:  The  singularly  perturbed  differential  game  defined  by 
Eqs . (71) - (75)  has  a  saddle  point  solution  characterized  by  the  triplet 
Ip* (« ,e) ,e* (• ,e) ,J* (• ,£) ]  in  a  closed  domain  V  of  the  state  space  Rn 
(J*  being  of  the  class  C*) . 

2.  Reduced  Game 

Let  the  reduced  order  game  be  defined  by 

x°  -  f (x°,yO,u°,vO,0)  ;  x(tQ)  *  XQ  (76) 

0  -  g{x°,yO,u°,vO,0)  (77) 

with  x°  €  Rm.  The  terminal  manifold  is 

i|>°[x°{tf),0]  =>  0  (78) 

The  cost  function  is  given  by 
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J  =  g[x°(tf),0]  (79) 

The  set  of  admissible  and  playable  strategy  pairs  for  this  game 
o  o 

p  (’,0)  and  e  (*,0)  are  such  that 

u°(t,0)  =  p° [x° (t,0) ,0]  u°  £  U 

(80) 

v°(t,0)  =  e° [x° (t,0) ,0]  v°  £  V 

transfer  the  system  to  the  terminal  manifold. 

Assumption  2:  The  reduced  order  game  defined  by  Eqs.  (76) - (80)  has  a 
saddle  point  solution,  characterized  by  the  triplet  [p*°( • ,0) ,e*° ( • ,0) 
J*°(x°,0)l  in  a  closed  domain  if  <=■  R™  .  (J*°being  C1)  . 

If  both  assumptions  (1  and  2)  hold,  it  can  be  asserted  that  for 
e  -*■  0  optimal  trajectories  of  both  games  approach  each  other 
everywhere,  except  for  the  fact  variables  near  to  t^  .  In  other  words: 

For  each  point  x*°(t,0)  on  the  optimal  trajectory  of  the  reduced 
game,  there  exists  a  point  x*(t,E)  on  the  projection  of  optimal  trajectory 
of  the  original  game  to  such  that 

x*(t,e)  =  x*° (t, 0)  +  0(E)  V  t  £  [tQ,tf]  (81) 

The  fast  variable  in  the  reduced  game  is  computed  from  Eq.(77) 
y*° (t, 0}  =  ip(x*°(t,0) ,u*° (t,0) v*° (t,0)  ] 


(82) 
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u1" (T,0)  »  pi[x0<yi(T,0)  ,0]  ,  u*  €  U 

i  i  i  i  (®7^ 

v  (t,0)  -  e  [xQ,y  (T ,0)  ,0]  ,  ,  v  €  v 

generate  a  solution  of  Eg. (85). 

Playability  of  the  boundary  layer  game  is  defined  to  guarantee 
asymptotic  matching.  A  pair  [p* (• ,0)e* (• ,0) ]  is  playable  if  it  leads 
a  trajectory  starting  at  yQ  to  the  isolated  equilibrium  point  y°(xQ) 
obtained  from  the  solution  of 

g[x0,y°<x0>,uo,v°]  -  0  (88) 

Moreover,  the  optimal  trajectory  has  to  satisfy 

lim  y*1 (T,0)  -  y°(xQ)  (89) 

Assumption  3;  The  boundary  layer  game  has  an  optimal  strategy  pair 
Ip*^ (* #0) ,e*^ (• ,0) ]  satisfying  (89). 

4.  Comgosite_Strate2£_Pair 

Supposing  that  Assumptions  1,  2  and  3  are  all  satisfied,  the 
following  zero-order  composite  strategy  pair  is  proposed  as  a  candidate 
for  the  original  singularly  perturbed  differential  game! 


u(t,0)  ■  p[x,y,0) 
v(t,0)  *  e[x,y,0] 


(90) 
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such  that  the  composite  control  functions  u,v  satisfy 
u(t,0)  -  u*°(t,0)  +  u*1  0^  -  CPu 

(91) 

v(t,0)  »  v*°(t,0)  +  v#i  ■ ,  0^  -  CPv 
for  all  t  €  ttQ,tf]  „ 

CP  and  CP  are  the  common  parts  of  the  reduced  order  and  boundary 
u  v 

layer  controls  cancelling  out  by  the  matching  process. 

The  proposed  strategy  pair  is  obviously  playable  and  can  serve  as 
a  suboptimal  approximation. 

5 .  Extended_Value 

Let  us  define  the  outcome  of  the  original  game  played  with  the 
composite  strategy  pair  proposed  in  Eq. (90)  as  the  Extended  Value  of  the 
game. 

J[x0,y0»p(*#e)  (**e)  J  «Gtx(tf),e]  ®  j£(x0,y0e)  (92) 

The  relationship  between  this  suboptimal  outcome  and  the  exact  optimal 
cost,  i.e.,  the  "Value"  of  the  game  J*(xQ,y0,e)  is  determined  by  the 
following  theorem. 

THEOREM  1.  Suppose  that  Assumptions  1,  2  and  3,  hold.  Then  the 
Extended  Value  of  a  singularly  perturbed  zero-sun  differential  game. 
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obtained  by  using  the  candldade  strategy  pair  of  Eq. (90) ,  is  bounded 
in  both  sides. 

J*txo'yo'E>  '  ^e<e)  -^e<xo'yo'e)  -  JMxo'yo'E)  +  VE)  (93) 

<p„(e),  and  ip  (E)  are  correction  terns  which  satisfy 
E  P 

lim  (G)  -  lira  ip  (£)  -  0  (94) 

£-*■0  E  £  -*•  0  p 

This  theorem,  proven  in  Ref.  [32],  and  illustrated  in  Fig.  8,  has  two 
direct  consequences, 

f 

COROLLARY  1.  The  Extended  Value  of  a  singularly  perturbed  zero-sum 
differential  game  satisfies  a  weak  saddle  inequality  expressed  by 

JtxQ.ygP.e.e)  -  ^(e)  <  J(x0,y0,p,e,£)  -  <  Jt*0'y0'p'®'E) 

i 

j 

+  lpp(£)  (95) 

which  is  a  combination  of  Eqs.(93)  and  (16). 

Substituting  Eq.  (94)  into  Eq.(93)  leads  to 

COROLLARY  2.  The  Extended  Value  oi  a  singularly  perturbed  zero-sun 
differential  game  tends  as  a  limit  towards  the  Value  of  the  game  as 
£  approaches  zero 


lim  J  (x  ,y Q»e)  -  lim 
£-*•0  £•*  0 


J*(x0,y0,e) 


(96) 
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C.  Application  to  Pursuit-Evasion  Games 

1.  Game  Characteristics 

A  class  of  frequently  used  pursuit-evasion  games  are  char¬ 
acterized  by  the  following: 

a.  There  is  a  time  scale  separation  between  the  variables  describing 
the  slow  relative  geometry  and  the  fast  variations  of  vehicle  dynamics. 

b.  The  dynamics  of  the  slow  variables  are  separately  controlled 
by  the  players. 

c.  The  dynamics  of  the  fast  variables  are  independent  of  the  slow 

ones . 

d.  The  fast  variables  are  scalars  and  independently  controlled  by 
the  players. 

e.  The  terminal  surface  is  defined  by  the  slow  variables  only. 

The  dynamic  equations  of  such  game  are 

x  -  fp(x,yp,u)  +  fE(x,yE,v)  ,  x(tQ)  -  xQ  (97) 

yp  -  9p(yp*u)  *  yp<V  -  yp  08) 


yE  "  Wv> 


'  VV  “  V 


x€Rn-2  ,  ypCR1  ,  y^R1  . 


,  v  6  V  c  R 


u  €  U  c  R* 
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Termination  of  the  game  (capture)  is  defined  by 

4>fx(tf)  ]  -  o  (loo) 

Let  the  pay-off  of  the  game  be  the  rime  of  capture  . 

The  Hamiltonian  of  the  game  is 


*  ■  1  ♦  *  V  *  Vp  *  Ve  11011 

where  X  ,  A  ,  A  are  the  respective  gradients  of  the  optimal  cost  de- 

X  PE 

te rained  by  the  adjoint  equations 

SX  ■  -  £  "  -  [ir  *  ari  !  W  -  v  9»a  *  v>0  (102) 

K  -  -^m  -  Xx35"‘  ’  w  ■ 0  11031 

.  m  _  3f_  3g_ 

X»  “  "  "Syl  m  ’  Xx  Hyl  ~  \  Hyl  f  W  '  °  (104) 

E  JE  E 

The  optimal  control  function  u* ,  v*  have  to  satisfy 


min  max  H  ■  0 
u€o  v  €  v 


(105) 
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2.  Transformation  to  a  Singularly  1'erturbed  Game 

In  many  cases  an  appropriate  transformation  will  lead  to 
define  a  small  parameter  £  multiplying  the  left  sides  of  Eqs.(9b)  and 
(99) .  If,  however,  the  time  scale  separation  is  obvious,  £  can  be 
inserted  artificially.  We  shall  pursue  this  forced  singular  perturba¬ 
tion  technique  (FSPT),  transforming  Eqs .  (97)  -  (99)  and  (102) -(104)  to 

*  =  fp(x,yp,u)  +  f£(x,yE,v)  x(tQ)  =  xQ  (106) 

£yp  *  gp(yp/u>  yp<V  =  yp  uo?) 

o 

£yE  =  WV)  yE{V  =  yE  (108) 

0 


perturbation  parameter. 
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4*  Boundary  Layer  Game 


Using  the  stretching  transformation  of  Eg. (70)  and  taking 
£  =  0  the  equations  of  this  game  are 


dx  -  v,  l 

——  *■  x  =  const  *  x 
dT  0 


«  gptyp^u1)  ,  yp(0)  -  yp 


(119) 


/  i  i ... 

dT~  *  9E  yE,V  }  yE<0) 


—r-  *  0-*  X1  *  const  ■  X°(x  )  »  X° 
dT  x  x  0  x_ 


3H _ AT  ffp  .  xi  S 

-  i  x  ~i  P  ~i 

3yp  3yp  3yp 


E  3  H 


,i 

x  .  i  E  3y 

3yE  yE 


It  can  be  seen  that  since  X^  is  constant,  determined  in  the  reduced 
game. the  boundary  layer  equations  of  each  player  can  be  solved  indepen¬ 
dently.  From  Eqs.  (119)  and  (122)  using  Eq. (105)  a  feedback  solution 
for  u*1  is  obtained 


.i  .i.  i  ,o  . 
u*  -  u*  (yp^XQ,x0) 


(124) 
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This  solution  has  to  satisfy  also  the  condition  of  asymptotic  stability 


y  + 
0 


)dT  -  y*  txQ) 


(125) 


Similarly,  the  optimal  control  of  the  evader  is  also  given  in  a  feed¬ 
back  form 


V1  -  Vlty‘,£  .«0) 
0 


(126) 


satisfying 


y  + 

E0 


\  9E(yE'V*i 


) dT  ■  y *  (xQ) 


(127) 


5,  Composite  Strategies 

let  us  consider  the  following  candidate  strategy  pair  as 
an  approximation  of  the  optimal  pair  of  the  gam  for  all  t  €  [tQ,t^] 


u(t)  »  p(x,yp) 
v (t)  *  e(x,yp) 


(128) 


such  that 


u(t)  ■  u*°tx(t)i  +  u#i[*0*yp  *  S(*#yp) 

v (t)  »  v*°[x(t)]  +  v*i[»0*yE  “  V*°(XQ)  •  v(x,yB) 


(129) 


I 
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In  this  particular  case  the  composite  control  is  identical  to  the  bound¬ 
ary  layer  control  replacing  the  constant  xQ  to  the  actual  value  of 
slow  variable  x  . 


u(x,yp)  -  u*  (x,yp) 


v(x,yp)  -  v*  (x,yp) 


(130) 


Using  this  uniformly  valid  feedback  strategy  pair  in  the  original  game 
(C  ■  1)  it  is  observed  that  the  termination  of  the  independent  boundary 
layers  may  not  coincide.  The  pursuer's  boundary  layer  terminates  at 
t  *  tp  determined  by 


V 


gptyp,u(x,yp)]dt  «  yp(x) 


(131) 


where  yp(x)  is  the  solution  of 


gptyp(x)*u°(x)]  -  o 


Similarly,  t  is  given  by 
£ 


(132) 


yE  + 
E0 


f  E 

J  VyE,v(x,yE)]dt  -  yci 


(x) 


(133) 


with 


gE[y°(x),v°(x)l  -  o 


(134) 


The  effectiveness  of  FSPT  in  non-linear  pursuit-evasion  games  is 
demonstrated  by  the  following  simple  example. 


-  -r  r  V  .  '  *  Vl  A  i- 
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D.  Example  of  Aircraft  vs  Aircraft  Interception  Game 
l.  0£i2i2ai_Probiem_Formuiation 

The  problem  of  interception  of  an  unoffensive  airplane  by 
an  offensive  one,  described  in  some  detail  in  subsection  III  B  2,  can  be 
characterized  by 

(i)  The  initial  range  of  separation  is  large  enough  to  allow  a 
"pure  pursuit"  phase. 

(ii)  The  pursuer  airplane  is  generally  equipped  with  a  missile  of 
larger  "capture  range"  than  the  turning  radius  of  the  air¬ 
planes.  Consequently,  the  "end-game"  phase  disappears. 

These  features  make  this  problem  to  be  a  specially  suitable  example 
for  SPT  application.  For  the  sake  of  simplicity,  a  constant  speed  two- 
dimensional  interception  will  be  considered.  The  method,  however,  is 
equally  applicable  for  variable  speed  and  three-dimensional  engagements. 

The  geometry  of  the  pursuit  in  a  plane  is  shown  in  Fig.  9  defining 

che  four  state  variables  (R,a,x_»X,y)  • 

P  'E 

The  equations  of  motion  are 

R  -  VE  cos  <XE-c)  -  Vp  cos  (Xp-O)  t  R(tQ)  -  Rq  (135) 

°  "  R  CVE  Sin  ^E-05  "  VP  Sin  (XE-o)1  *  <J<tQ)  -  aQ  (136) 

••  xp<v  -  xp 

0 


xp  -  v1  *  M  i. 1 


(137) 
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*E  =V  ;  |  v  |  <  1  ;  XE(t0)=XEQ  (138) 

a)?  ,  cje  are  the  maximal  turning  rates  of  the  pursuer  and  evader  respecti¬ 
vely. 

The  game  terminates  when  the  range  between  the  players  becomes  equal 
to  the  capture  radius  £ 

R(tf)  -  £  ;  R(tf)  <  0  (139) 

The  pay-off  to  be  optimized  is  the  time  of  "capture"  defined  by 
Eq. (139) 

J  *  t  (140) 

Capture  is  guaranteed  if  Vp  >  VE  and  ap  »  (iipVp  >  ^  »  o^V^,  . 

He  assume  that  these  conditions  are  satisfied.  Hie  Hamiltonian  of  the 
problem  is 

H  -  1  +  Ar[Ve  cosO^-O)  -  Vp  costXp-O)]  + 

A 

+  -2.  tvE  sintXg-O)  -  Vp  sinlXp-O)]  +  Apupu  +  A^v  (141) 

This  problem  belongs  to  the  class  of  pursuit-evasion  games  analyzed 
in  the  previous  subsection.  Its  exact  solution  was  obtained  by  Simakova, 
more  than  a  decade  ago  [49]  and  will  be  used  for  comparison.  In  Ref.  [32] 
the  system  equations  were  transformed  to  a  genuine  singularly  perturbed 


structure  with 
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P 


Cd  R 

P  0 


( 142) 


In  this  subsection  the  KEPT  version  will  be  solved  demonstrating  the 
equivalence  of  the  two  techniques  for  the  zero-order  approximation. 


2.  Forced  Singular  Perturbation  Model 

Since  the  rate  of  turn  of  the  line  of  sight  0  is  much  slower 
than  the  turning  rates  of  the  participating  airplanes,  xp  and  XE  can 
be  considered  as  "fast"  variables. 

The  equations  of  this  forced  singularly  perturbed  dynamic  system  are 

R  *= 

o  = 

e*p 

exE 

The  set  of  the  adjoint  equations  have  the  form 

=  '  fi  =  ~2  [VE  Sln  %~a)  ~  VP  Sin  (XP"a)  (147) 

R 

•  dH 

Xa=  ~  X R  [VP  Sin  <XP_a)  ■  VE  Sin  (XE'0)  + 

X 

+  —  [  V  cos  (X  -a)  -  V  cos  (x  -O)  ]  ;  A  (t  )  =0  (148) 

R  t  P  P  0  1 


E  COS  -  VP 

cos  (Xp-a) 

■R(V 

=  Ro 

(143) 

'  lvE  sin  (XE-o)  _ 

VP  sin  (Xp-o)l 

o(t0) 

O 

D 

It 

(144) 

(DpU 

XP(V 

=  Xpo 

(145) 

V 

XE  ^0* 

=  Xeh 

(146) 
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M  .  Aa 

*  ■  -  XR  VP  sln  (V0’  ♦  T  VP  008  ,Xp-°>  ■ 


Xp(tf)  -  0 

3H  X 

-  Sjfe  *  \  VE  Sin  (XE“0)  "  T  VE  008  lXE“0)  * 


(149) 


w  - 0 


(150) 


The  conditions  of  optimality  which  require  Min  Max  H  *  0 

u  v 


yield 


sign  Xp  Xr  y  0 


v*  -  sign  X  X  ft  0 

£  E 


(151) 

(152) 


K*  *»  o 


(153) 


The  missing  condition  for  XR(tf)  can  be  obtained  from  Eqs.(141)  and 
(153) 


VV  “  1/tVp  COS  (Va)f  “  VE  COS  (XE"°)f1 


(154) 


3.  Reduced  Game 


In  the  reduced  game  (  setting  e  »  0)  we  have  from  Eqs.(145) 


and  (146) 


u*°  -  0 


(155) 


v* 


0 


(156) 
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a;  (t  >  tj)  -  o 


(190) 


(T  >  TJ)  -  0 


(191) 


satisfying  the  requirements  of  matching  with  the  reduced  game. 

The  optimal  boundary  layer  controls  can  thus  be  expressed  by  substi¬ 
tution  of  Eqs . (184)  and  (185)  into  Eqs. (177)  and  (178)  in  a  feedback 
form 


u*1 (T)  «  u*[Xp]  -  -sign 


v*1^)  -  v*  (Xgl  -  -sign  (Xg-a°) 


(192) 

(193) 


5 .  Zero  Order  Composite  Strategies 

According  to  previous  discussion  in  subsection  V  C,  we  propose 
the  following  uniformly  valid  strategy  pair  {£>(•)*£(•)}  as  a  suboptimal 
candidate  for  the  original  game 


u(t)  »  -  sign  (Xp-O) 
u(t)  -  u 


v (t)  »  -  sign  (Xg-O) 


v  (t)  -  v_ 


Xp  +  <3 

Xp  -  a 


XE  *  0 


Y  *  c 

ae 


(194) 


(195) 


te  singular  controls  u  and  v  are  such  that 

as 


‘Xp*.  -  IX,).  -  ® 


(196) 
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(197) 

(198) 

This  zero-order  composite  FSPT  strategy,  expressed  by  Eqs.(194)- 
(198)  in  a  feedback  form,  is  identical  to  the  solution  obtained  using 
a  genuine  singular  perturbation  model  [32]. 

This  strategy  consists  for  each  player  of  three  subarcs:  (i)  "hard" 
turn  until  the  velocity  vector  is  aligned  with  the  3ine  of  sight; 

(ii)  line  of  sight  guidance,  if  the  other  player  is  still  in  phase  (i) ; 

(iii)  a  straight  line  dash  until  capture.  The  resulting  trajectories 
for  different  initial  conditions  are  shown  in  Figs.  11  and  12. 


yielding 


u  =  — —  sin  (X„-<7) 
s  RU)  E 


~  *VP 

v  -  - —  sin  (y -a) 
s  Rul,  AP 


6.  Comgarison_to_the  Exact_Solution 

In  the  exact  solution  of  Simakova  [49],  shown  in  broken  lines 
in  Figs.  11  and  12,  the  direction  of  the  final  dash  is  the  common  tangent 
of  the  players  turning  circles,  determined  by  the  initial  conditions.  The 
Optimal  strategy  for  each  player  is  to  align  the  velocity  vector  with 
this  common  tangent. 

If  the  initial  conditions  are  such  that  both  players  finish  their 
hard  turns  simultaneously,  the  trajectories  of  the  optimal  and  the  SPT 
solutions,  and  consequently  the  outcomes  of  the  game,  are  identical. 


For  any  other  initial  conditions  one  of  the  players  completes  its  "hard" 
turn  earlier  and  following  the  subop timai  strategy  of  phase  (ii)  de¬ 
viates  from  the  optimal  one.  If  it  is  the  pursuer,  the  time  of  capture 
will  be  slightly  longer  than  in  the  optimal  game  (see  Fig.  11).  If  the 
evader  reaches  first  the  line  of  sight  direction  (see  Fig.  12),  the 
capture  time  of  the  SPT  solution  will  be  shorter  than  its  value  predicted 
by  Simakova  (493-  The  differences  are,  however,  very  small.  A  quantita¬ 
tive  comparison  (32]  has  shown  that  for  large  initial  ranges,  relative 
to  the  radius  of  turn  of  the  airplanes  (£p  in  Eq.  (142)  less  than  0.3), 
the  differences  are  negligible  (less  than  1%) . 

The  usefulness  of  the  SPT  can  be  appreciated  by  this  excellent  ac¬ 
curacy,  in  addition  to  the  simple  method  of  solution  yielding  feedback 
control  laws.  Moreover,  the  suboptimal  SPT  strategies  can  be  easily 
implemented.  They  are  based  only  on  line  of  sight  measurements  (direction 
or  rate)  but  do  not  require  range  information  or  the  knowledge  of  the 
opponents  flight  direction.  Note  also  that  in  this  example  the  validity 
of  Assumptions  1-3  cam  be  directly  verified. 


VI  CONCLUDING  REMARKS 


In  this  Chapter  two  analytical  approaches,  aimed  to  solve  perfect 
information  zero-sum  differential  games,  were  presented  and  applied  to 

v 

suitably  formulated  simple  pursuit-evasion  examples.  It  was  shown  that 
both  methods  —  (i)  a  linear  one  (LDG)  and  (ii)  a  technique  based  on 

the  concept  of  singular  perturbations  (SPT)  —  have  a  definite  potential 
to  yield  closed  form  solutions  for  properly  modelled  "near  real  world" 
air  combat  problems.  For  sake  of  illustrative  clarity  the  selected 
examples  were  of  simplified  nature  (constant  speed,  first  order  time 
constant  or,  in  one  case,  even  two-dimensional  geometry).  However,  the 
extension  of  these  methods  for  more  complex  problems,  such  as  variable 
speed,  high  order  transfer  functions,  three-dimensional  motion,  currently 
under  investigation  [50],  does  not  seem  to  present  any  difficulty. 

It  has  to  be  admitted  that  each  of  the  solution  techniques  exhibit 

some  inherent  limitation.  A  linear  mathematical  model  cannot  validly 

i 

describe  pursuit-evasion  problems  involving  large  changes  of  the  inter¬ 
ception  geometry,  which  frequently  occur  in  the  initial  phase.  At  the 
other  end,  the  singular  perturbation  approach  is  unable  to  provide  a 
"feedback"  solution  for  engagements  of  rapidly  varying  terminal  phase. 

In  order  to  solve  pursuit-evasion  problems,  in  which  the  initial 
acquisition  and  the  "end-game"  are  of  equal  importance,  an  appropriate 
combination  of  both  methods  has  to  be  investigated.  It  is  hoped  that 
the  successful  application  of  the  individual  solution  techniques  demon¬ 
strated  in  this  Chapter  will  encourage  such  endeavour. 
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